?This codebook.txt file was generated on 20190215 by Teresa Auch Schultz


-------------------
GENERAL INFORMATION
-------------------


1. Title of Dataset: Accessibility of Open Educational Resources
2. Author Information


  Principal Investigator Contact Information
        Name:Teresa Auch Schultz
           Institution: University of Nevada, Reno
           Address: Room 214H, Mathewson-IGT Knowledge Center, University of Nevada, Reno, 1664 N. Virgnia St., Reno NV
           Email: teresas@unr.edu


3. Date of data collection (single date, range, approximate date) 2020_01_01 to 2020_06_30

4. Geographic location of data collection (where was data collected?): Reno, Nevada, United States


5. Information about funding sources that supported the collection of the data: Association of College and Research Libraries Research and Scholarly Environment Committee ($5,000).




--------------------------
SHARING/ACCESS INFORMATION
-------------------------- 


1. Licenses/restrictions placed on the data: None


2. Links to publications that cite or use the data:


3. Links to other publicly accessible locations of the data:


4. Links/relationships to ancillary data sets:


5. Was data derived from another source? No
           If yes, list source(s): 

6. Recommended citation for the data: 




---------------------
DATA & FILE OVERVIEW
---------------------


1. File List
   A. Filename: OER_Accessibility_Dataset.csv      
      Short description:   Quantitative data from an accessibility evaluation based on a modified form of WCAG 2.1 of 355 open textbooks.
   B. Filename: OER_Accessibility_Dataset.xlsx      
      Short description:   Quantitative data from an accessibility evaluation based on a modified form of WCAG 2.1 of 355 open textbooks. Includes links between each category to the WCAG 2.1 element it reflects.
       

2. Relationship between files:  They are the same file but in different formats. The Excel copy includes hyperlinks to the books and for each variable name to the WCAG 2.1 element it reflects.      




3. Additional related data collected that was not included in the current data package: 




4. Are there multiple versions of the dataset? No
   If yes, list versions:
           Name of file that was updated:
                     i. Why was the file updated? 
                ii. When was the file updated?
           Name of file that was updated:
                      i. Why was the file updated?
                    ii. When was the file updated?






--------------------------
METHODOLOGICAL INFORMATION
--------------------------


1. Description of methods used for collection/generation of data: 
Researches used the SUNY Geneseo's OASIS aggregator to find open textbooks for the study sample. We searched the database using the "textbook" filter. We then randomly chose a number between 1 and 10, which turned out to be 7. We then started with the 7th book in the results list and then chose every fifth book, which went through most of the results. In some instances, the URL took the researchers to a group of books; when this happened, we chose the fifth book listed. This resulted in 393 books. However, some were excluded from the study because of issues with the book no longer being available, the item was not actually a textbook, or the item was not in one of the main file formats being studied (website/HTML, EPUB, PDF, or Word file). This resulted in a final study sample of 355 books.

We then crafted a modified rubric based on WCAG 2.1 to evaluate the books for accessibility. We were primarily interested in content under the control of the textbook author and thus did not include parts of WCAG that are more under the purview of a web designer. We also excluded any components related to audio or video in the belief that most open textbooks still follow the traditional textbook format of text and images. We also focused on A and AA level components, although we did include one AAA category (ordered headings). This resulted in 16 categories, although not all categories applied to all file formats. We created this rubric in consultation with an accessibility expert on our campus. A written out version of the rubric can be found at https://bit.ly/OERaccessibility.

The two lead researchers tested the rubric several times with our student research assistant, reaching 86% interrater agreement. The student researcher then coded the remaining textbooks, with a review of all codes by the lead researchers on a weekly basis to issue a group ruling whenever there was any question on any of the variables.

We evaluated just the first 20 pages or 10,000 words of a book for accessibility. We did not count how many times a book failed a category; it was marked as a fail if we found any fails or as a pass if we did not find any fails. If a category did not apply to a book (i.e. there were no tables in the first 20 pages), we marked it as "NA" for not applicable. We used the following tools to assist us in evaluating the books: Siteimprove for websites, Ace by DAISY and Calibre for EPUBs, Adobe Acrobat Pro for PDFs, Microsoft Word's built-in accessibility checker, and the Paciello Group's Colour Contrast Analyzer on all documents. We also manually inspected the code as needed for websites and EPUB files and also used the NVDA or VoiceOver screen reader on all documents whenever we were unsure if an item could be properly interpreted by a screen reader. 

Notes about how we opted to pass/fail certain elements in a book:
1. Because of citation rules, if a URL was typed out as part of reference list, we did not count that as a fail
2. We did not fail an empty header unless it appeared to be a cheat to get around the ordered headings rule.
3. We did not evaluate any content that appeared to be part of an authoring/hosting platform, vs. the actual content of the book.
4. For a PDF to be counted as tagged, all 20 pages we evaluated had to be tagged.
5. We used a low bar for link text as well as heading text. As long as it gave some idea to the included content, the item passed.

2. Methods for processing the data: An Excel sheet was used to anaylze for descriptive data.

3. Instrument- or software-specific information needed to interpret the data: Any program that can use a CSV file


4. Standards and calibration information, if appropriate: NA


5. Environmental/experimental conditions:  


6. Describe any quality-assurance procedures performed on the data:


7. People involved with sample collection, processing, analysis and/or submission: Three researchers collected the data: the primary contact, Elena Azadbakht, and Jay Arellano.






-----------------------------------------
DATA-SPECIFIC INFORMATION FOR: [FILENAME]
-----------------------------------------


1. Number of variables: 19


2. Number of cases/rows: 355




3. Missing data codes: NA
        Code/symbol        Definition
        Code/symbol        Definition


4. Variable List


    A. Name: Unique ID
             Description: A unique number assigned to each case.

    B. Name: Book Title
       Description: The title of the open textbook under review

    C. Name: Book URL
       Description: The URL for the open textbook as listed by OASIS.

    D. Name: Non-text content has a text alternative or marked as null/decorative?
       Description: Did the book include alternative text for images or other non-text content or mark the image as decorative so that screen readers would skip over it?
		Pass, Fail, Not applicable
                    
    E. Name: All elements approrpriately marked with HTML codes/tags (i.e. paragraph, links, strong, etc.)
       Description: Did the website use appropriate formatting for each element, such as using heading tags for headers instead of using the paragraph tag and relying on bold/italics to make it look as a header?
		Pass, Fail, Not applicable

    F. Name: Does not use i, bold, and other incorrect codes.
       Description: The book does not rely on outdated HTML codes, such as i for italics and that can cause problems for screen readers, but instead uses accepted coding, such as <em> for emphasis
                Pass, Fail, Not applicable

    G. Name: Information is not only conveyed in the formatting of text or other visual cues?
       Description: This mostly pertains to graphs, where the only distinction between lines or bars might be colors, which might not be distinguishable for some people. 
             Pass, Fail, Not applicable

    H. Name: For non-HTML, file is tagged at all?
       Description: Was the item tagged at all (not including whether tags were appropriate? Tags are needed for screen readers to decipher between various elements, such as paragraphs, headings, lists, etc.
             Pass, Fail, Not applicable

    I. Name: Tables are appropriately tagged/coded?
       Description: Included ensuring that tables were not used incorrectly, such as for formatting.
		Pass, Fail, Not applicable
                    

    J. Name: Lists are appropriately tagged/coded?
       Description: The item used the list formatting feature to mark a list, not just manually inserting dashes to make a list.
                Pass, Fail, Not applicable

    K. Name: Correctly uses heading coding/tags
       Description: This is focused on the correct ordering and nesting of heading labels. Heading 2 should always follow a Heading 1, a Heading 3 should always come after a Heading 2, and on.
		Pass, Fail, Not applicable

    L. Name: Links are appropriately tagged so that they are recognized when you tab through them (PDF only)
       Description: Essentially, does a PDF reader recognize a link and enables the user to click on the link to go to it.
               Pass, Fail, Not applicable

    M. Name: Is this content ordered to make sense?
       Description: If a user tabs through a document, does it tab through in sequential order or does it skip around?
              Pass, Fail, Not applicable

    N. Name: Color contrast is appropriate?
       Description: The color contrast meets the accepted ratio by WCAG 2.1
              Pass, Fail

    O. Name: Does not rely on images of text except as needed? 
       Description: Books should not include images of text unless needed, and if they do, should provide a transcription of any pertinent next needed to understand the context of the image as part of the alternative text.
             Pass, Fail, Not applicable

    P. Name: Has title and title is appropriate for the content?
       Description: The document has a set title that accurately reflects the content of the book.
            Pass, Fail
                    
    Q. Name: Uses descriptive link text?
       Description: The book does not write out full URLs (unless short) and instead hyper links text that reflects where the link will send the reader (i.e. CNN is hyperlinked when directing the reader to the CNN.com website).
	     Pass, Fail, Not applicable
                   
    R. Name: Uses descriptive headings and labels?
       Description: Any headings used should give some idea to the content below it. The bar was low. 
		Pass, Fail, Not applicable
                    

    S. Name: A language is set for the document/website and changes of language are also indicated
       Description: Coding is included that indicates what language the document is written in.

    T. Book File Format
       Description: The file format of the book
		epub, HTML, PDF, WORD

    